Line of sight assistance device and method

ABSTRACT

An assistance device enabling communication with a wearable device includes: a memory; and a processor including hardware, the processor being configured to acquire line-of-sight information about a line of sight of a user wearing the wearable device, based on the line-of-sight information, generate blind region state information indicating a state of a blind region shielded by a shielding object in a visual field region of the user, and output the blind region state information to the wearable device.

CROSS-REFERENCE TO RELATED APPLICATION(S)

The present application claims priority to and incorporates by referencethe entire contents of Japanese Patent Application No. 2018-231164 filedin Japan on Dec. 10, 2018.

BACKGROUND 1. Technical Field

The present disclosure relates to an assistance device, an assistancemethod, and a computer-readable recording medium.

2. Related Art

A technique for displaying on a mobile phone an image around a vehicleseen from a viewpoint of a user in the vehicle is known (for example,refer to JP 2013-162328 A). In this technique, when the user in thevehicle aims the mobile phone, an image in which the vehicle istransparent in a direction in which the user aims the mobile phone isdisplayed on a display of the mobile phone.

SUMMARY

Meanwhile, along with development of high-speed and high-capacitycommunication, a device that can acquire a large amount of informationvia a network has been manufactured. Under such circumstances, it isexpected that the device can make the user wearing the deviceintuitively grasp a danger hidden in a surrounding environment of theuser based on information that the device has acquired by means ofcommunication.

The disclosure is accomplished by taking such matters as mentioned aboveinto consideration thereof, and it is desirable to provide an assistancedevice, an assistance method, and a computer-readable recording mediumenabling a user to intuitively grasp a surrounding environment.

In some embodiments, provided is an assistance device enablingcommunication with a wearable device. The assistance device includes: amemory; and a processor including hardware, the processor beingconfigured to acquire line-of-sight information about a line of sight ofa user wearing the wearable device, based on the line-of-sightinformation, generate blind region state information indicating a stateof a blind region shielded by a shielding object in a visual fieldregion of the user, and output the blind region state information to thewearable device.

In some embodiments, provided is an assistance device configured to beworn by a user. The assistance device includes: a memory; and aprocessor including hardware. The processor is configured to acquireline-of-sight information about a line of sight of the user, based onthe line-of-sight information, generate blind region state informationindicating a state of a blind region shielded by a shielding object in avisual field region of the user, and output the blind region stateinformation.

In some embodiments, provided is an assistance method performed by anassistance device enabling communication with a wearable device. Theassistance method includes: acquiring line-of-sight information about aline of sight of a user wearing the wearable device, based on theline-of-sight information read out from a memory, generating blindregion state information indicating a state of a blind region shieldedby a shielding object in a visual field region of the user, andoutputting the blind region state information to the wearable device.

In some embodiments, provided is a non-transitory computer-readablerecording medium with an executable program stored thereon. The programcauses an assistance device enabling communication with a wearabledevice to execute: acquiring line-of-sight information about a line ofsight of a user wearing the wearable device; based on the line-of-sightinformation, generating blind region state information indicating astate of a blind region shielded by a shielding object in a visual fieldregion of the user; and outputting the blind region state information tothe wearable device.

The above and other objects, features, advantages and technical andindustrial significance of this disclosure will be better understood byreading the following detailed description of presently preferredembodiments of the disclosure, when considered in connection with theaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic view illustrating a schematic configuration of anassistance system according to a first embodiment;

FIG. 2 illustrates a schematic configuration of a wearable deviceaccording to the first embodiment;

FIG. 3 is a block diagram illustrating a functional configuration of thewearable device according to the first embodiment;

FIG. 4 is a flowchart illustrating an overview of processing executed bythe wearable device according to the first embodiment;

FIG. 5 schematically illustrates an example of a shielding object;

FIG. 6 schematically illustrates an example of blind region stateinformation according to the first embodiment;

FIG. 7 is a flowchart illustrating an overview of processing executed bythe wearable device according to a second embodiment;

FIG. 8 schematically illustrates an example of blind region stateinformation according to the second embodiment;

FIG. 9 is a flowchart illustrating an overview of processing executed bythe wearable device according to a third embodiment;

FIG. 10 schematically illustrates an example of blind region stateinformation according to the third embodiment;

FIG. 11 schematically illustrates an example of blind region stateinformation that a control unit outputs to a projection unit accordingto a modification example of the third embodiment;

FIG. 12 illustrates a schematic configuration of an assistance systemaccording to a fourth embodiment;

FIG. 13 is a block diagram illustrating a functional configuration ofthe assistance system according to the fourth embodiment;

FIG. 14 is a flowchart illustrating an overview of processing executedby an assistance device according to the fourth embodiment;

FIG. 15 schematically illustrates an example of blind region stateinformation according to the fourth embodiment;

FIG. 16 schematically illustrates another example of blind region stateinformation according to the fourth embodiment;

FIG. 17 schematically illustrates another example of blind region stateinformation according to the fourth embodiment;

FIG. 18 illustrates an example of a shape in an image corresponding toblind region state information according to the fourth embodiment;

FIG. 19 illustrates an example of another shape in an imagecorresponding to blind region state information according to the fourthembodiment;

FIG. 20 illustrates an example of another shape in an imagecorresponding to blind region state information according to the fourthembodiment;

FIG. 21 is a schematic view illustrating a schematic configuration of anassistance system according to a fifth embodiment;

FIG. 22 illustrates a schematic configuration of a wearable deviceaccording to another embodiment;

FIG. 23 illustrates a schematic configuration of a wearable deviceaccording to another embodiment;

FIG. 24 illustrates a schematic configuration of a wearable deviceaccording to another embodiment;

FIG. 25 illustrates a schematic configuration of a wearable deviceaccording to another embodiment; and

FIG. 26 illustrates a schematic configuration of a wearable deviceaccording to another embodiment.

DETAILED DESCRIPTION

Hereinbelow, embodiments of the disclosure will be described withreference to the drawings. Note that the disclosure is not limited tothe following embodiments. Also, in the following description, identicalcomponents are labeled with the same reference signs.

First Embodiment

Configuration of Assistance System

FIG. 1 is a schematic view illustrating a schematic configuration of anassistance system according to a first embodiment. FIG. 2 illustrates aschematic configuration of a wearable device according to the firstembodiment. FIG. 3 is a block diagram illustrating a functionalconfiguration of the wearable device according to the first embodiment.

An assistance system 1000 illustrated in FIGS. 1 to 3 includes awearable device 1 that a user U1 can wear and a server 2. The wearabledevice 1 and the server 2 are configured to enable mutual informationcommunication via a base station 3 and a network 4. Also, the wearabledevice 1 receives a signal from a plurality of global positioning system(GPS) satellites 5 and calculates a position of the wearable device 1itself based on the received signal. Also, the server 2 acquires via thebase station 3 and the network 4 image data generated when the GPSsatellite 5 captures from the air an image of the user U1 wearing thewearable device 1 and a surrounding area. Note that, in the firstembodiment, the wearable device 1 functions as an assistance device.

Configuration of Wearable Device

First, a configuration of the wearable device 1 will be described. Asillustrated in FIGS. 1 to 3, the wearable device 1 includes an imagecapturing unit 11, a nine-axis sensor 12, a line-of-sight sensor 13, aprojection unit 14, a global positioning system (GPS) sensor 15, awearing sensor 16, a communication unit 17, and a control unit 18.Although the nine-axis sensor 12 is used in the present example, thesensor may function less in a case in which a three-axis or six-axissensor is sufficient.

Under control of the control unit 18, the image capturing unit 11captures an image along a line of sight of a user, for example, togenerate image data and outputs the image data to the control unit 18.The image capturing unit 11 includes an optical system including one ora plurality of lens(es) and a charge coupled device (CCD), acomplementary metal oxide semiconductor (CMOS), or the likelight-receiving an object image obtained by converging light by means ofthe optical system to generate image data. As illustrated in FIG. 2, aplurality of image capturing units 11 may be provided in the wearabledevice 1.

The nine-axis sensor 12 includes a three-axis gyro sensor, a three-axisacceleration sensor, and a three-axis geomagnetic sensor (compass). Thenine-axis sensor 12 detects angular velocity and acceleration generatedin the wearable device 1 and outputs the detection results to thecontrol unit 18. The nine-axis sensor 12 also detects geomagnetism todetect an absolute direction and outputs the detection result to thecontrol unit 18.

The line-of-sight sensor 13 detects a direction of a line of sight of adriver, who is a wearer of the wearable device 1, and outputs thedetection result to the control unit 18. The line-of-sight sensor 13includes an optical system, a CCD or a CMOS, a memory, and a processorincluding hardware such as a CPU and a GPU. With use of known templatematching, for example, the line-of-sight sensor 13 detects an unmovingpart of a driver's eye (for example, the inner corner of the eye) as areference point and a moving part of the eye (for example, the iris) asa moving point and detects the direction of the line of sight of thedriver based on a positional relationship between the reference pointand the moving point.

The projection unit 14 projects image, video, and character informationto a display unit (for example, a lens unit) of the wearable device orthe retina of the driver under control of the control unit 18. Theprojection unit 14 includes RGB laser beams emitting R, G, and B laserbeams, a MEMS mirror reflecting the laser beams, a reflection mirrorprojecting the laser beams reflected on the MEMS mirror to the retina ofthe driver, and the like.

The GPS sensor 15 calculates positional information about a position ofthe wearable device 1 based on signals received from the plurality ofGPS satellites and outputs the calculated positional information to thecontrol unit 18. The GPS sensor 15 includes a GPS reception sensor andthe like.

The wearing sensor 16 detects a user's wearing state and outputs thedetection result to the control unit 18. The wearing sensor 16 includesa pressure sensor detecting pressure when the user wears the wearabledevice 1, a vital sensor detecting vital information such as a bodytemperature, a pulse, brain waves, blood pressure, and a sweating state,and the like.

The communication unit 17 transmits various information to the server 2and receives various information from the server 2 via the network 4 inconformity to predetermined communication standard under control of thecontrol unit 18. The communication unit 17 includes a communicationmodule enabling wireless communication.

The control unit 18 controls operations of the respective units includedin the wearable device 1. The control unit 18 includes a memory and aprocessor including hardware such as a CPU. The control unit 18 acquiresfrom the line-of-sight sensor 13 line-of-sight information about a lineof sight of the user U1 wearing the wearable device 1 and, based on theline-of-sight information, generates and outputs to the projection unit14 blind region state information indicating a state of a blind regionshielded by a shielding object in a visual field region of the user U1.Meanwhile, in the first embodiment, the control unit 18 functions as aprocessor.

Configuration of Server

Next, a configuration of the server 2 will be described. The server 2includes a communication unit 201 and a control unit 202.

The communication unit 201 transmits various information and receivesvarious information via the network 4 and the base station 3 inconformity to predetermined communication standard under control of thecontrol unit 202. The communication unit 201 also transmits variousinformation to the wearable device 1 and receives various informationfrom the GPS satellite 5 and the wearable device 1 in conformity topredetermined communication standard under control of the control unit202. The communication unit 201 includes a communication module enablingwireless communication.

The control unit 202 includes a memory and a processor including somesort of hardware such as a CPU. The control unit 202 controls operationsof the respective units included in the server 2.

Processing of Wearable Device

Next, processing executed by the wearable device 1 will be described.FIG. 4 is a flowchart illustrating an overview of processing executed bythe wearable device 1.

As illustrated in FIG. 4, the control unit 18 first determines whetheror not the user U1 wears the wearable device 1 based on a detectionresult input from the wearing sensor 16 (Step S101). In a case in whichit is determined by the control unit 18 that the user U1 wears thewearable device 1 (Step S101: Yes), the wearable device 1 moves to StepS102 described below.

Subsequently, the control unit 18 acquires positional information of thewearable device 1 detected by the GPS sensor 15 (Step S102) andtransmits the positional information of the wearable device 1 via thecommunication unit 17 to the server 2 (Step S103).

The control unit 18 then acquires image data generated by the imagecapturing unit 11 and line-of-sight information about a line of sight ofthe user U1 from the line-of-sight sensor 13 (Step S104).

Subsequently, the control unit 18 determines whether or not the user U1stares at a shielding object in a visual field region for apredetermined staring period or longer based on the image data generatedby the image capturing unit 11 and the line-of-sight informationacquired from the line-of-sight sensor 13 (Step S105). Specifically, asillustrated in FIG. 5, the control unit 18 detects a staring region of ashielding object Q1 which the user U1 stares at based on the image datagenerated by the image capturing unit 11 and the line-of-sightinformation acquired from the line-of-sight sensor 13. The control unit18 then determines whether or not the staring period for which the userU1 stares at the staring region of the shielding object Q1 is apredetermined period or longer, such as one second to two seconds orlonger. Here, the shielding object Q1 is a wall, a member, or the likeshielding a blind region that the user U1 cannot visually recognize.Also, the blind region is a region or a space which is shielded by theshielding object Q1 and which the user. U1 cannot visually recognize.Also, the predetermined period can arbitrarily be changed in accordancewith the operation of the user U1 and the walking speed of the user U1.In a case in which it is determined by the control unit 18 that thestaring period for which the user U1 stares at the shielding object isthe predetermined period or longer (Step S105: Yes), the wearable device1 moves to Step S106 described below. Conversely, in a case in which itis determined by the control unit 18 that the staring period for whichthe user U1 stares at the shielding object is less than thepredetermined period (Step S105: No), the wearable device 1 moves toStep S108 described below.

In Step S106, the control unit 18 acquires blind region stateinformation indicating a state of the blind region shielded by theshielding object in the visual field region of the user U1 from theserver 2 via the communication unit 17. Specifically, the control unit18 acquires from the server 2 image data in the visual field directionof the user U1 defined by a detection result detected by the nine-axissensor 12, that is, current image data captured from the air a currentposition of the wearable device 1 detected by the GPS sensor 15. Here,the current image data is image data acquired by the GPS satellite 5,for example.

Subsequently, the control unit 18 outputs the blind region stateinformation to the projection unit 14 so that the blind region stateinformation may be displayed at a region corresponding to the line ofsight of the user U1 (Step S107). Specifically, as illustrated in FIG.6, the control unit 18 outputs to the projection unit 14 an imagecorresponding to the image data acquired from the server 2 so that blindregion state information A1 may be displayed at the staring region ofthe shielding object Q1. Accordingly, since the blind region stateinformation A1 is an image for the current state, the user U1 canintuitively grasp a state of the blind region behind the shieldingobject Q1.

Subsequently, the control unit 18 determines whether or not the user U1takes off the wearable device 1 to end safety assistance by means of thewearable device 1 (Step S108). In a case in which it is determined bythe control unit 18 that the user U1 ends the safety assistance by meansof the wearable device 1 (Step S108: Yes), the wearable device 1 endsthe processing. Conversely, in a case in which it is determined by thecontrol unit 18 that the user U1 does not end the safety assistance bymeans of the wearable device 1 (Step S108: No), the wearable device 1returns to Step S101 described above.

According to the first embodiment described above, since the controlunit 18 outputs the image data acquired from the server 2 to theprojection unit 14 so that the blind region state information A1 may bedisplayed at the shielding object Q1, the user U1 can intuitively graspthe state of the blind region behind the shielding object Q1.

Also, according to the first embodiment, since the control unit 18outputs the blind region state information to the projection unit 14 sothat the blind region state information may be displayed at the regioncorresponding to the line of sight of the user U1, and the blind regionstate information A1 is the image for the current state, the user U1 canintuitively grasp the state of the blind region behind the shieldingobject Q1.

Also, according to the first embodiment, in a case in which the staringperiod for which the user U1 stares at the shielding object Q1 is thepredetermined period or longer, the control unit 18 outputs the blindregion state information. Thus, the user can grasp the state of theblind region only when the user desires.

Second Embodiment

Next, a second embodiment will be described. An assistance systemaccording to the second embodiment has an equal configuration to that ofthe assistance system 1000 according to the first embodiment and differsin terms of processing executed by the wearable device. Processingexecuted by the wearable device will be described below. Note thatidentical components to those in the assistance system 1000 according tothe first embodiment described above are labeled with the same referencesigns, and detailed description of the duplicate components is omitted.

Processing of Wearable Device

FIG. 7 is a flowchart illustrating an overview of processing executed bythe wearable device 1 according to the second embodiment. In FIG. 7,Steps S201 to S206 correspond to Steps S101 to S106 described above inFIG. 4, respectively.

In Step S207, the control unit 18 performs known pattern matching or thelike to the image data acquired from the server 2 to determine whetheror not there exists an object in the blind region. For example, thecontrol unit 18 performs pattern matching or the like to an imagecorresponding to the image data acquired from the server 2 to determinewhether or not there exists an object such as a person, a car, abicycle, or an animal. In a case in which it is determined by thecontrol unit 18 that there exists an object in the blind region (StepS207: Yes), the wearable device 1 moves to Step S208 described below.Conversely, in a case in which it is determined by the control unit 18that there exists no object in the blind region (Step S207: No), thewearable device 1 moves to Step S209 described below.

In Step S208, the control unit 18 outputs blind region state informationto the projection unit 14 so that the blind region state information maybe displayed at the staring region of the shielding object Q1.Specifically, as illustrated in FIG. 8, the control unit 18 outputsblind region state information A2 and A21 to the projection unit 14 sothat blind region state information A2 and A21 may be displayed at thestaring region of the shielding object Q1. Accordingly, since the userU1 can intuitively grasp the blind region state information A2 and A21,the user U1 can grasp existence of an object in the blind region behindthe shielding object Q1. Meanwhile, although the arrow icon and the markicon are output as the blind region state information A2 and A21 in FIG.8, a graphic, a symbol, a color, a message, or the like corresponding tothe object, such as a person-shaped icon or a message in a case of aperson, may be output instead of the arrow and the mark. As a matter ofcourse, the control unit 18 may output a sound by means of anot-illustrated loudspeaker or the like at the same time as output ofthe blind region state information A2.

Subsequently, the control unit 18 determines whether or not the user U1takes off the wearable device 1 to end safety assistance by means of thewearable device 1 (Step S209). In a case in which it is determined bythe control unit 18 that the user U1 ends the safety assistance by meansof the wearable device 1. (Step S209: Yes), the wearable device 1 endsthe processing. Conversely, in a case in which it is determined by thecontrol unit 18 that the user U1 does not end the safety assistance bymeans of the wearable device 1 (Step S209: No), the wearable device 1returns to Step S201 described above.

According to the second embodiment described above, since the controlunit 18 outputs the blind region state information A2 to the projectionunit 14 so that the blind region state information A2 may be displayedat the staring region of the shielding object Q1, the user U1 canintuitively grasp the blind region state information A2 and can thusgrasp existence of an object in the blind region behind the shieldingobject Q1.

Third Embodiment

Next, a third embodiment will be described. An assistance systemaccording to the third embodiment has an equal configuration to that ofthe assistance system 1000 according to the first embodiment and differsin terms of processing executed by the wearable device. Processingexecuted by the wearable device will be described below. Note thatidentical components to those in the assistance system 1000 according tothe first embodiment described above are labeled with the same referencesigns, and detailed description of the duplicate components is omitted.

Processing of Wearable Device

FIG. 9 is a flowchart illustrating an overview of processing executed bythe wearable device 1 according to the third embodiment. In FIG. 9,Steps S301 to S306 correspond to Steps S101 to S106 described above inFIG. 4, respectively.

In Step S307, the control unit 18 performs known pattern matching or thelike to the image data acquired from the server 2 to determine whetheror not there exists an object in the blind region. In a case in which itis determined by the control unit 18 that there exists an object in theblind region (Step S307: Yes), the wearable device 1 moves to Step S308described below. Conversely, in a case in which it is determined by thecontrol unit 18 that there exists no object in the blind region (StepS307: No), the wearable device 1 moves to Step S312 described below.

In Step S308, the control unit 18 determines whether or not the objectexisting in the blind region is a moving object. Specifically, thecontrol unit 18 uses at least two chronologically-adjacent image datapieces acquired from the server 2 to calculate the moving amount of theobject, such as a moving vector of the object, and determines whether ornot the calculated moving amount is a predetermined value or higher.Here, the moving object is an object moving at certain speed such as achild, an animal, a pedestrian, a bicycle, a motorcycle, and a vehicle.In a case in which it is determined by the control unit 18 that theobject existing in the blind region is a moving object (Step S308: Yes),the wearable device 1 moves to Step S309 described below. Conversely, ina case in which it is determined by the control unit 18 that the objectexisting in the blind region is not a moving object (Step S308: No), thewearable device 1 moves to Step S310 described below.

In Step S309, the control unit 18 outputs blind region state informationcorresponding to the moving object to the projection unit 14 based onthe image data generated by the image capturing unit 11 and the imagedata acquired from the server 2. Specifically, as illustrated in FIG.10, the control unit 18 outputs to the projection unit 14 a syntheticimage corresponding to synthetic image data obtained by synthesizing theimage data generated by the image capturing unit 11 with the image dataof the blind region acquired from the server 2 at a predetermined ratio(for example, 5:5) so that the synthetic image may be displayed at thestaring region of the shielding object Q1. Accordingly, since the blindregion behind the shielding object Q1 is displayed in a virtuallysemi-transmissive state, the user U1 can grasp the current state of theblind region without feeling strange. Also, in a case in which thecontrol unit 18 detects a moving object H1 such as a child as a resultof known pattern matching or the like to the image data of the blindregion acquired from the server 2, the control unit 18 outputs to theprojection unit 14 marks M1 and M2 so that the marks M1 and M2 may bedisplayed around the moving object H1. At this time, the control unit 18calculates speed of the moving object H1 and walking speed of the userU1 wearing the wearable device 1 with use of at least twochronologically-adjacent image data pieces acquired from the server 2and calculates the degree of urgency based on relative speed between themoving object H1 and the wearable device 1 and traveling directions ofthe moving object H1 and the user U1 wearing the wearable device 1. Thecontrol unit 18 then outputs the marks M1 and M2 to the projection unit14 so that ways to display the marks M1 and M2 may be changed inaccordance with the degree of urgency, e.g., so that the marks M1 and M2may be displayed in red or yellow in accordance with the degree ofurgency. Accordingly, since the user U1 can intuitively grasp blindregion state information A3, the user U1 can grasp a state of the movingobject H1 hidden behind the shielding object Q1. Further, since the userU1 can grasp speed of the moving object H1 based on the states of themarks M1 and M2, the user U1 can predict time until an encounter withthe moving object H1. Note that, although the control unit 18 performsthe synthesis at a synthetic ratio of 5:5 in FIG. 10, the syntheticratio is not limited to this and can arbitrarily be changed. Also, thecontrol unit 18 is not required to express the moving object H1 in theblind region state information A3 accurately and may express the movingobject H1 with use of a simple graphic, such as an icon and an avatar,in accordance with processing speed for image generation. After StepS309, the wearable device 1 moves to Step S312 described below.

In Step S310, the control unit 18 outputs blind region state informationcorresponding to a still object to the projection unit 14 based on theimage data acquired from the server 2. Specifically, the control unit 18outputs the blind region state information A2 (refer to FIG. 8 describedabove) to the projection unit 14 so that the blind region stateinformation A2 may be displayed at a region corresponding to theshielding object Q1. Accordingly, since the user U1 can intuitivelygrasp the blind region state information A2, the user U1 can graspexistence of an object in the blind region behind the shielding objectQ1. After Step S310, the wearable device 1 moves to Step S312 describedbelow.

Subsequently, the control unit 18 determines whether or not the user U1takes off the wearable device 1 to end safety assistance by means of thewearable device 1 (Step S312). In a case in which it is determined bythe control unit 18 that the user U1 ends the safety assistance by meansof the wearable device 1 (Step S312: Yes), the wearable device 1 endsthe processing. Conversely, in a case in which it is determined by thecontrol unit 18 that the user U1 does not end the safety assistance bymeans of the wearable device 1 (Step S312: No), the wearable device 1returns to Step S301 described above.

According to the third embodiment described above, since the blindregion behind the shielding object Q1 is displayed in a virtuallysemi-transmissive state, the user U1 can intuitively grasp the blindregion state information A3, and the user U1 can thus grasp the state ofthe moving object H1 hidden behind the shielding object Q1 withoutfeeling strange.

Also, according to the third embodiment, since the control unit 18outputs the marks M1 and M2 to the projection unit 14 so that ways todisplay the marks M1 and M2 may be changed in accordance with therelative speed between the moving object H1 and the wearable device 1,the user U1 can intuitively grasp the blind region state information A3,and the user U1 can thus grasp the state of the moving object H1 hiddenbehind the shielding object Q1.

Also, according to the third embodiment, since the control unit 18generates blind region state information with use of image data andoutputs the blind region state information to the wearable device 1, theuser can grasp the current state in the blind region.

Modification Example of Third Embodiment

Next, a modification example of the third embodiment will be described.FIG. 11 schematically illustrates an example of blind region stateinformation that the control unit 18 outputs to the projection unit 14according to the modification example of the third embodiment.

As illustrated in FIG. 11, the control unit 18 generates image dataincluding an opening portion obtained by virtually hollowing apredetermined region including the staring region of the shieldingobject Q1 based on the image data for the blind region acquired from theserver 2 and the line-of-sight information detected by the line-of-sightsensor 13 and outputs an image corresponding to the image data as blindregion state information A4 to the projection unit 14. Accordingly,since the blind region behind the shielding object Q1 is displayed in astate in which the region including the staring region of the shieldingobject Q1 which the user U1 stares at is hollowed, the user U1 can graspthe current state of the blind region without feeling strange. Further,the control unit 18 may calculate the degree of urgency based onrelative speed between a moving object H2 and the user U1 wearing thewearable device 1 and traveling directions of the moving object H2 andthe user U1 wearing the wearable device 1 and circle the opening portionwith an arbitrary color in accordance with the degree of urgency, e.g.,with green, yellow, or red in accordance with the degree of urgency.

According to the modification example of the third embodiment describedabove, since the blind region behind the shielding object Q1 isdisplayed in a state in which the staring region of the shielding objectQ1 which the user U1 stares at is hollowed, the user U1 can grasp thecurrent state of the blind region without feeling strange.

Fourth Embodiment

Next, a fourth embodiment will be described. In the aforementioned firstto third embodiments, the wearable device 1 acquires image data via thenetwork 4 from the server 2. In an assistance system according to thefourth embodiment, an electronic control unit (ECU) provided in avehicle outputs blind region state information to the wearable device incooperation with the wearable device, and the wearable device projectsand displays the blind region state information. Note that identicalcomponents to those in the assistance system 1000 according to the firstembodiment described above are labeled with the same reference signs,and detailed description of the duplicate components is omitted.

Configuration of Assistance System

FIG. 12 illustrates a schematic configuration of the assistance systemaccording to the fourth embodiment. FIG. 13 is a block diagramillustrating a functional configuration of the assistance systemaccording to the fourth embodiment.

An assistance system 1001 illustrated in FIGS. 12 and 13 includes thewearable device 1 and an assistance device 6 performing two-waycommunication with the wearable device 1 in conformity to predeterminedcommunication standard.

Configuration of Assistance Device

Next, a configuration of the assistance device 6 will be described. Theassistance device 6 illustrated in FIGS. 12 and 13 is mounted in avehicle 600 and assists safety of a driver in the vehicle 600 at thetime of driving and at the time of parking in cooperation with anotherECU mounted in the vehicle 600. The assistance device 6 includes anignition switch 60 (hereinbelow referred to as “an IG switch 60”), avehicle speed sensor 61, a line-of-sight sensor 62, an image capturingunit 63, a communication unit 64, a car navigation system 65, and an ECU66.

The IG switch 60 accepts start and stop of electric systems such as anengine and a motor. The IG switch 60 starts an IG power supply in a casein which the IG switch 60 is in an on state and stops the IG powersupply in a case in which the IG switch 60 is in an off state.

The vehicle speed sensor 61 detects vehicle speed when the vehicle 600is running and outputs the detection result to the ECU 66.

The line-of-sight sensor 62 detects a line of sight of the driver andoutputs the detection result to the ECU 66. The line-of-sight sensor 62includes an optical system, a CCD or a CMOS, a memory, and a processorincluding hardware such as a CPU and a GPU. With use of known templatematching, for example, the line-of-sight sensor 62 detects an unmovingpart of a driver's eye (for example, the inner corner of the eye) as areference point and a moving part of the eye (for example, the iris) asa moving point and detects the line of sight of the driver based on apositional relationship between the reference point and the movingpoint. Note that, although the line of sight of the driver is detectedusing a visible camera as the line-of-sight sensor 62 in the fourthembodiment, the disclosure is not limited to this, and the line of sightof the driver may be detected using an infrared camera. In a case inwhich the infrared camera is used as the line-of-sight sensor 62,infrared light is emitted to the driver by means of an infrared lightemitting diode (LED) or the like, a reference point (for example, thecorneal reflection) and a moving point (for example, the pupil) aredetected from image data generated by capturing an image of the driverwith use of the infrared camera, and the line of sight of the driver isdetected based on a positional relationship between the reference pointand the moving point.

As illustrated in FIG. 12, a plurality of image capturing units 63 areprovided outside the vehicle 600 at three or more locations including atleast a front lateral side, a rear side, and both lateral sides so thatan image capturing viewing angle may be 360°, for example. Under controlof the ECU 66, the image capturing unit 63 captures an image of aperiphery of the vehicle 600 to generate image data and outputs theimage data to the ECU 66. The image capturing unit 63 includes anoptical system including one or a plurality of lens(es) and a CCD, aCMOS, or the like light-receiving an object image obtained by converginglight by means of the optical system to generate image data.

The communication unit 64 transmits various information to the wearabledevice 1, another vehicle, a user terminal device, or the like andreceives various information from the wearable device 1, anothervehicle, the user terminal device, or the like in conformity topredetermined communication standard under control of the ECU 66. Thecommunication unit 64 also transmits various information to anot-illustrated server and receives various information from the servervia a network in conformity to predetermined communication standardunder control of the ECU 66. The communication unit 64 includes acommunication module enabling wireless communication.

The car navigation system 65 includes a GPS sensor 651, a map database652, and a notification unit 653.

The GPS sensor 651 receives a signal from a plurality of GPS satellitesand transmission antennae and calculates a position of the vehicle 600based on the received signal. The GPS sensor 651 includes a GPSreception sensor and the like. Note that a plurality of GPS sensors 651may be mounted to improve direction accuracy of the vehicle 600.

The map database 652 stores various kinds of map data. The map database652 may include a recording medium such as a hard disk drive (HDD) and asolid state drive (SSD).

The notification unit 653 may include a display unit 653 a displayingimage, video, and character information, a sound output unit 653 bgenerating a sound such as a voice and a warning sound, a conductionunit conducting a sound by means of bone conduction, and the like. Thedisplay unit 653 a includes a display such as a liquid crystal displayand an organic electroluminescence (EL) display. The sound output unit653 b includes a loudspeaker.

The car navigation system 65 configured as above superimposes a currentposition of the vehicle 600 acquired by the GPS sensor 651 on the mapdata stored in the map database 652 to notify the driver of informationincluding a road on which the vehicle 600 is currently traveling, aroute to a destination, and the like by means of the display unit 653 aand the sound output unit 653 b.

The ECU 66 controls operations of the respective units included in theassistance device 6. The ECU 66 includes a memory and a processorincluding hardware such as a CPU. The ECU 66 acquires line-of-sightinformation about a line of sight of the user U1 wearing the wearabledevice 1 and, based on the line-of-sight information, generates andoutputs to the projection unit 14 blind region state informationindicating a state of a blind region shielded by a shielding object in avisual field region of the user U1. Meanwhile, in the fourth embodiment,the ECU 66 functions as a processor.

Processing of Assistance Device

Next, processing executed by the assistance device 6 will be described.FIG. 14 is a flowchart illustrating an overview of processing executedby the assistance device 6.

As illustrated in FIG. 14, the ECU 66 determines whether or not the userU1 has ridden the vehicle 600 (Step S401). For example, the ECU 66calculates a distance between the vehicle 600 and the wearable device 1based on positional information detected by the GPS sensor 15 of thewearable device 1 via the communication unit 64 and positionalinformation detected by the GPS sensor 651 of the car navigation system65 to determine whether or not the calculated distance is less than apredetermined value and determines that the user U1 has ridden thevehicle 600 in a case in which the distance is less than thepredetermined value. In a case in which it is determined by the ECU 66that the driver has ridden the vehicle 600 (Step S401: Yes), theassistance device 6 moves to Step S402 described below. Conversely, in acase in which it is determined by the ECU 66 that the user U1 has notridden the vehicle 600 (Step S401: No), the assistance device 6 ends theprocessing.

In Step S402, the ECU 66 determines whether or not the user U1 wears thewearable device 1. Specifically, the ECU 66 receives a wearing signalindicating a detection result from the wearing sensor 16 of the wearabledevice 1 via the communication unit 64 and determines whether or not theuser U1 wears the wearable device 1 based on the received wearingsignal. In a case in which it is determined by the ECU 66 that the userU1 wears the wearable device 1 (Step S402: Yes), the assistance device 6moves to Step S403 described below. Conversely, in a case in which it isdetermined by the ECU 66 that the user U1 does not wear the wearabledevice 1 (Step S402: No), the assistance device 6 moves to Step S407described below.

In Step S403, the ECU 66 acquires vehicle speed information aboutvehicle speed of the vehicle 600 from the vehicle speed sensor 61,line-of-sight information about a line of sight of the user U1 ridingthe vehicle 600 from the line-of-sight sensor 62, and image data fromthe image capturing unit 63.

Subsequently, the ECU 66 determines based on the vehicle speedinformation acquired from the vehicle speed sensor 61 whether or not thevehicle speed of the vehicle 600 is equal to or less than predeterminedspeed (Step S404). For example, the ECU 66 determines whether or not thevehicle speed of the vehicle 600 is 10 km/h. Note that the predeterminedspeed can arbitrarily be set. In a case in which it is determined by theECU 66 that the vehicle speed of the vehicle 600 is equal to or lessthan the predetermined speed (Step S404: Yes), the assistance device 6moves to Step S405 described below. Conversely, in a case in which it isdetermined by the ECU 66 that the vehicle speed of the vehicle 600 isnot equal to or less than the predetermined speed (Step S404: No), theassistance device 6 moves to Step S410 described below.

In Step S405, the ECU 66 acquires line-of-sight information about a lineof sight of the user U1 from the line-of-sight sensor 62.

Subsequently, the ECU 66 performs known pattern matching or the like toan image corresponding to the image data acquired from the imagecapturing unit 63 to determine whether or not there exists an object ina blind region of the vehicle 600 (Step S406). In a case in which it isdetermined by the ECU 66 that there exists an object in the blind regionof the vehicle 600 (Step S406: Yes), the assistance device 6 moves toStep S407 described below. Conversely, in a case in which it isdetermined by the ECU 66 that there exists no object in the blind regionof the vehicle 600 (Step S406: No), the assistance device 6 moves toStep S411 described below.

In Step S407, the ECU 66 determines whether or not the object existingin the blind region is a moving object. In a case in which it isdetermined by the ECU 66 that the object existing in the blind region isa moving object (Step S407: Yes), the assistance device 6 moves to StepS408 described below. Conversely, in a case in which it is determined bythe ECU 66 that the object existing in the blind region is not a movingobject (Step S407: No), the assistance device 6 moves to Step S409described below.

In Step S408, the ECU 66 generates and outputs blind region stateinformation corresponding to the moving object based on the image dataof the blind region acquired from the image capturing unit 63.Specifically, as illustrated in FIG. 15, the ECU 66 generates image dataincluding an opening portion obtained by virtually hollowing a region ofthe blind region in which a moving object H10 exists, such as apredetermined region including a portion of shielding objects Q10 andQ11 such as an instrument panel and a front pillar. The ECU 66 thenoutputs blind region state information A10 corresponding to the imagedata to the wearable device 1. Accordingly, since the blind region stateinformation A10 behind the shielding object Q10 is displayed, the userU1 can intuitively grasp a state of the blind region without feelingstrange. In this case, the ECU 66 calculates speed of the moving objectwith use of at least two chronologically-adjacent image data piecesacquired from the image capturing unit 63, acquires speed of the vehicle600 acquired from the vehicle speed sensor 61, calculates relative speedbetween the moving object and the vehicle 600 (subject vehicle) andtraveling directions of the moving object and the vehicle 600, andcalculates the degree of urgency based on the relative speed between themoving object and the vehicle 600 (subject vehicle) and the travelingdirections of the moving object and the vehicle 600. Based on the degreeof urgency, the ECU 66 may output the aforementioned marks M1 and M2 inaccordance with the degree of urgency as in FIG. 10 to the wearabledevice 1 so that the marks M1 and M2 may be displayed. After Step S408,the assistance device 6 moves to Step S410 described below.

In Step S409, the ECU 66 generates and outputs blind region stateinformation corresponding to a still object based on the image data ofthe blind region acquired from the image capturing unit 63.Specifically, the ECU 66 generates blind region state information (forexample, the icon in FIG. 8 or the like) corresponding to the stillobject based on the image data of the blind region acquired from theimage capturing unit 63. The ECU 66 then outputs the blind region stateinformation to the wearable device 1 so that the blind region stateinformation may be displayed at a region of the shielding object Q10 atwhich the object exists. After Step S409, the assistance device 6 movesto Step S410 described below.

In Step S410, in a case in which it is determined by the ECU 66 that theIG switch 60 is turned off to end driving of the vehicle 600 (Step S410:Yes), the assistance device 6 ends the processing. Conversely, in a casein which it is determined by the ECU 66 that the IG switch 60 is notturned off, and that driving of the vehicle 600 is not ended (Step S410:No), the assistance device 6 returns to Step S402 described above.

In Step S411, the ECU 66 determines based on the line-of-sightinformation about the line of sight of the user U1 from theline-of-sight sensor 62 whether or not the staring period for which theuser U1 stares at a predetermined region of the vehicle 600, such as thefront pillar, is a predetermined period (for example, two seconds) orlonger (Step S411). In a case in which it is determined by the ECU 66that the staring period for which the user U1 stares at thepredetermined region of the vehicle 600 is the predetermined period orlonger (Step S411: Yes), the assistance device 6 moves to Step S412described below. Conversely, in a case in which it is determined by theECU 66 that the staring period for which the user U1 stares at thepredetermined region of the vehicle 600 is not the predetermined periodor longer (Step S411: No), the assistance device 6 moves to Step S413described below.

In Step S412, the ECU 66 outputs image data acquired from the imagecapturing unit 63 via the communication unit 64 as blind region stateinformation to the wearable device 1 to project and display the blindregion state information at the blind region behind the shielding objectof the vehicle 600 on the wearable device 1. Specifically, similarly toFIG. 15 described above, the ECU 66 generates image data including anopening portion obtained by virtually hollowing the staring region ofthe shielding objects Q10 and Q11. The ECU 66 then outputs the blindregion state information A10 corresponding to the image data to thewearable device 1 so that the blind region state information A10 may bedisplayed at the staring region at which the user U1 stares.Accordingly, since the blind region state information A10 behind theshielding object Q10 is displayed, the user U1 can intuitively grasp astate of the blind region without feeling strange. After Step S412, theassistance device 6 moves to Step S410.

In Step S413, the ECU 66 determines based on the line-of-sightinformation about the line of sight of the user U1 from theline-of-sight sensor 62 whether or not the line of sight of the user ismoving. In a case in which it is determined by the ECU 66 that the lineof sight of the user is moving (Step S413: Yes), the assistance device 6moves to Step S414 described below. Conversely, in a case in which it isdetermined by the ECU 66 that the line of sight of the user is notmoving (Step S413: No), the assistance device 6 moves to Step S410.

In Step S414, the ECU 66 determines whether or not the wearable device 1is outputting the blind region state information. In a case in which itis determined by the ECU 66 that the wearable device 1 is outputting theblind region state information (Step S414: Yes), the assistance device 6moves to Step S415 described below. Conversely, in a case in which it isdetermined by the ECU 66 that the wearable device 1 is not outputtingthe blind region state information (Step S414: No), the assistancedevice 6 moves to Step S410.

In Step S415, the ECU 66 causes the wearable device 1 to output theblind region state information for a predetermined period. Specifically,as illustrated in FIG. 16, in a case in which the wearable device 1 isoutputting blind region state information A10, A11, and A12, and even ina case in which the line of sight of the user U1 moves, the ECU 66 keepsoutputting the image data so that the blind region state informationA10, A11, and A12 may be projected and displayed on the wearable device1 for a predetermined period such as for five seconds. Accordingly, evenin a case in which the user U1 moves the line of sight, the user U1 canreliably prevent moving objects and objects from being overlooked. AfterStep S415, the assistance device 6 moves to Step S410.

According to the fourth embodiment described above, since, the ECU 66outputs to the wearable device 1 the blind region state information A10corresponding to the image data including the opening portion obtainedby virtually hollowing the staring region of the shielding object Q10 tocause the blind region state information A10 behind the shielding objectQ10 or the shielding object Q11 to be displayed in a state in which thestaring region which the user U1 stares at is hollowed, the user U1 canintuitively grasp the state of the blind region without feeling strange.

Also, according to the fourth embodiment, in a case in which the vehiclespeed of the vehicle 600 is equal to or less than the predeterminedspeed, the ECU 66 outputs the blind region state information A10 to thewearable device 1. This can prevent the processing from being performedunnecessarily and prevent the user from feeling strange.

Meanwhile, although the ECU 66 outputs to the wearable device 1 theimage data including the opening portion obtained by virtually hollowingthe shielding object as the blind region state information in the fourthembodiment, the shielding object may be in a see-through state, forexample. Specifically, as illustrated in FIG. 17, the ECU 66 acquiresrespective image data pieces from the image capturing unit 11 of thewearable device 1 and the image capturing unit 63 and, with use of thetwo image data pieces, outputs to the wearable device 1 image dataobtained by virtually seeing through the shielding objects in thevehicle 600 such as an instrument panel and a front pillar.Subsequently, the wearable device 1 causes the projection unit 14 toproject and display an image corresponding to the image data, receivedfrom the assistance device 6, obtained by virtually seeing through theshielding objects in the vehicle 600 such as an instrument panel and afront pillar. Consequently, since the shielding objects in the vehicle600 are in a see-through state, the user U1 can intuitively grasp aposition of an object.

Also, although the ECU 66 outputs to the wearable device 1 the imagedata including the opening portion obtained by virtually hollowing theshielding object in the fourth embodiment, the hollowing shape canarbitrarily be changed. For example, as illustrated in FIG. 18, the ECU66 may generate image data including an opening portion O1 obtained byvirtually hollowing a shielding object Q100 located a predetermineddistance L1 away from the wearable device 1 in a columnar shape andoutput the image data to the wearable device 1. Also, as illustrated inFIG. 19, the ECU 66 may generate image data obtained by virtuallyhollowing the shielding object Q100 located the predetermined distanceL1 away from the wearable device 1 in a conical shape O2 and output theimage data to the wearable device 1. Further, as illustrated in FIG. 20,the ECU 66 may generate image data obtained by virtually hollowing theshielding object Q100 located the predetermined distance L1 away fromthe wearable device 1 in a spherical shape O3 and output the image datato the wearable device 1.

Also, although the ECU 66 projects and displays an image having apredetermined size on the wearable device 1 in the fourth embodiment,the size of the image to be projected and displayed on the wearabledevice 1 may be changed in accordance with a pinch operation, in whichthe distance between the thumb and the index finger is shifted from adistance D1 to a distance D2, as illustrated in FIG. 21, for example.

Fifth Embodiment

Next, a fifth embodiment will be described. In the aforementioned fourthembodiment, the ECU 66 serving as an assistance device transmits blindregion state information to the wearable device 1. However, in the fifthembodiment, a server transmits blind region state information to thewearable device. Note that identical components to those in theassistance system 1001 according to the fourth embodiment describedabove are labeled with the same reference signs, and detaileddescription of the duplicate components is omitted.

FIG. 22 is a schematic view illustrating a schematic configuration of anassistance system according to a fifth embodiment. An assistance system1002 illustrated in FIG. 22 includes the wearable device 1 which theuser U1 wears, a server 2A, and assistance devices 6 respectivelymounted on a plurality of vehicles 600. The wearable device 1, theserver 2A, and the plurality of assistance devices 6 are configured toenable mutual information communication via the base station 3 and thenetwork 4.

Configuration of Server

Next, a configuration of the server 2A will be described. The server 2Aincludes the communication unit 201 and a control unit 202A.

The control unit 202A includes a memory and a processor including somesort of hardware such as a CPU, a GPU, an FPGA, a DSP, and an ASIC. Thecontrol unit 202A acquires line-of-sight information about a line ofsight of the user wearing the wearable device 1 via the communicationunit 201 and, based on the line-of-sight information, generates andoutputs to the wearable device 1 blind region state informationindicating a state of a blind region shielded by a shielding object in avisual field region of the user. The control unit 202A functionssimilarly to the control unit 18 according to the aforementioned firstto third embodiments or the ECU 66 according to the aforementionedfourth embodiment. Meanwhile, in the fifth embodiment, the control unit202A functions as a processor.

According to the fifth embodiment described above, since the controlunit 202A outputs blind region state information to the wearable device1 so that the blind region state information may be displayed at astaring region of the shielding object, the user U1 can intuitivelygrasp the blind region state information and can thus grasp existence ofan object in the blind region behind the shielding object.

Other Embodiments

Although examples of using the glasses-type wearable device 1 which thedriver can wear have been described in the first to fifth embodiments,the disclosure is not limited to this and can be applied to variouswearable devices. For example, as illustrated in FIG. 23, the disclosurecan be applied to a contact-lenses-type wearable device 1A having animage capturing function. Also, the disclosure can be applied to awearable device 1B in FIG. 24 or a brain-chip-type wearable device 1C inFIG. 25, which is a device transmitting information directly to thebrain of the user U1. Further, as a wearable device 1D in FIG. 26, ahelmet-like device including a visor may be used. In this case, in thewearable device 1D, an image may be projected and displayed on thevisor.

Also, in the first to fifth embodiments, although the wearable device 1projects an image to the retina of the driver to let the driver visuallyrecognize the image, the image may be projected and displayed on a lensof glasses or the like.

Also, in the first to fifth embodiments, the aforementioned “units” canbe replaced with “circuits”. For example, the control unit can bereplaced with the control circuit.

Also, a program to be executed by the assistance device according toeach of the first to fifth embodiments is provided by recordinginstallable or executable file data on a computer-readable recordingmedium such as a CD-ROM, a flexible disk (FD), a CD-R, a digitalversatile disk (DVD), a USB medium, and a flash memory.

Also, the program to be executed by the assistance device according toeach of the first to fifth embodiments may be provided by storing theprogram on a computer connected to a network such as the Internet anddownloading the program via the network.

Meanwhile, in the description of the flowcharts in the presentspecification, although the expressions “first”, “then”, “subsequently”,and the like are used to clarify a processing order of the steps, theprocessing order required to carry out each of the present embodimentsshall not be defined uniquely by these expressions. That is, theprocessing order in each of the flowcharts described in the presentspecification can be changed unless it is inconsistent.

Although several embodiments of the present application have beendescribed above in detail with reference to the drawings, theseembodiments are illustrative only, and the disclosure can be embodied inother ways by modifying or improving the embodiments described in thesection of the disclosure of the disclosure in various ways based onknowledge of those skilled in the art.

According to the disclosure, since the processor outputs the blindregion state information to the wearable device so as to display theblind region state information in a region corresponding to the line ofsight of the user, the user can grasp a state of the blind regionintuitively.

According to the disclosure, since the processor outputs the blindregion state information when there exists the object, the user cangrasp that there exists an object.

According to the disclosure, since the processor outputs the blindregion state information corresponding to the moving object in the blindregion, the user can grasp a state of the moving object approaching theblind region.

According to the disclosure, since the processor outputs the blindregion state information in accordance with relative speed between themoving object in the blind region and the assistance device, the usercan grasp a state of the moving object approaching the blind region inreal time.

According to the disclosure, since the processor generates and outputsthe blind region state information with use of the image data pieceobtained by capturing an image of the blind region, the user can grasp acurrent state in the blind region.

According to the disclosure, since the blind region state information isoutput to the staring region at which the user stares, the user cangrasp a state of the blind region that the user needs.

According to the disclosure, in a case in which the staring period ofthe user is the predetermined period or longer, the processor outputsthe blind region state information. Thus, the user can grasp a state ofthe blind region only when the user desires.

According to the disclosure, since the synthetic image in which theshielding object is in a semi-transmissive state is output as the blindregion state information, the user can grasp a state of the blind regionwithout feeling strange.

According to the disclosure, since the processor outputs the imagecorresponding to the image data piece obtained by capturing an image ofthe blind region as the blind region state information, the user cangrasp an actual state of the blind region.

According to the disclosure, since the processor outputs the imageincluding the opening portion as the blind region state information, theuser can grasp a state of the blind region without feeling strange.

According to the disclosure, since the processor outputs the blindregion state information only at the time of low speed or a stop, thiscan prevent the processing from being performed unnecessarily andprevent the user from feeling strange.

According to the disclosure, since blind region state informationindicating a state of a blind region shielded by a shielding objectwhich is blind as seen in a visual field region of a user is output,this exerts an effect of enabling the user to intuitively grasp asurrounding environment.

Although the disclosure has been described with respect to specificembodiments for a complete and clear disclosure, the appended claims arenot to be thus limited but are to be construed as embodying allmodifications and alternative constructions that may occur to oneskilled in the art that fairly fall within the basic teaching herein setforth.

What is claimed is:
 1. An assistance device enabling communication witha wearable device, comprising: a memory; and a processor includinghardware, the processor being configured to: acquire line-of-sightinformation about a line of sight of a user wearing the wearable devicethat includes a projector configured to emit light to the user; acquirean image data piece obtained by capturing an image of a blind regionshielded by a shielding object in a visual field region of the user, theimage data piece being one of a plurality of image data pieces;determine, based on the image data piece, whether an object exists inthe blind region; calculate, based on a determination that an objectexists in the blind region, a speed at which the object is moving basedon the plurality of image data pieces; generate, based on theline-of-sight information and on a relative speed between the movingobject and the assistance device, blind region state informationincluding a degree of urgency indicating a time to an encounter betweenthe object and the user; and output the blind region state informationto the projector of the wearable device, wherein the output blind regionstate information includes a first mark and a second mark, each of thefirst mark and the second mark being projected in a region of thewearable device corresponding to a location of the shielding object,wherein the first mark indicates less urgency and extends a firstdistance out from the region corresponding to the location of theshielding object, and wherein the second mark indicates more urgency andextends a second distance out from the region corresponding to thelocation of the shielding object, the second distance being less thanthe first distance.
 2. The assistance device according to claim 1,wherein the image data piece is one of a plurality ofchronologically-adjacent image data pieces, and the processor is furtherconfigured to: sequentially acquire the plurality ofchronologically-adjacent image data pieces, and based on a determinationthat the object exists in the blind region, determine whether the objectis moving based on the plurality of image data pieces.
 3. The assistancedevice according to claim 1, wherein the processor is further configuredto: generate the blind region state information with use of the imagedata piece.
 4. The assistance device according to claim 1, wherein theprocessor is further configured to: acquire an image data piece obtainedby capturing an image of the shielding object, generate synthetic imagedata obtained by synthesizing the image data piece obtained by capturingthe image of the shielding object with the image data piece obtained bycapturing the image of the blind region at a predetermined ratio, andoutput a synthetic image corresponding to the synthetic image data asthe blind region state information to the wearable device.
 5. Theassistance device according to claim 1, wherein the processor is furtherconfigured to: output an image corresponding to the image data piece asthe blind region state information to the wearable device.
 6. Theassistance device according to claim 5, wherein the processor is furtherconfigured to: generate an image including an opening portion obtainedby virtually hollowing the shielding object in a columnar, conical, orspherical shape based on an image data piece obtained by capturing animage of the shielding object and the image data piece obtained bycapturing the image of the blind region, and output the image as theblind region state information to the wearable device.
 7. The assistancedevice according to claim 1, wherein the processor is further configuredto: generate the blind region state information by using an image datapiece captured from a position above the wearable device.
 8. Theassistance device according to claim 1, wherein the blind region stateinformation includes a color indicating the degree of urgency.
 9. Anassistance device configured to be worn by a user, comprising: a memory;and a processor including hardware, wherein the processor is configuredto: acquire line-of-sight information about a line of sight of the userwearing a wearable device that includes a projector configured to emitlight to the user, acquire an image data piece obtained by capturing animage of a blind region shielded by a shielding object in a visual fieldregion of the user, the image data piece being one of a plurality ofimage data pieces; determine, based on the image data piece, whether anobject exists in the blind region; calculate, based on a determinationthat an object exists in the blind region, a speed at which the objectis moving based on the plurality of image data pieces; generate, basedon the line-of-sight information and on a relative speed between themoving object and the assistance device, blind region state informationincluding a degree of urgency indicating a time to an encounter betweenthe object and the user, and output the blind region state informationto the projector of the wearable device, wherein the output blind regionstate information includes a first mark and a second mark, each of thefirst mark and the second mark being projected in a region of thewearable device corresponding to a location of the shielding object,wherein the first mark indicates less urgency and extends a firstdistance out from the region corresponding to the location of theshielding object, and wherein the second mark indicates more urgency andextends a second distance out from the region corresponding to thelocation of the shielding object, the second distance being less thanthe first distance.
 10. An assistance method performed by an assistancedevice enabling communication with a wearable device, comprising:acquiring line-of-sight information about a line of sight of a userwearing the wearable device that includes a projector configured to emitlight to the user; acquiring an image data piece obtained by capturingan image of a blind region shielded by a shielding object in a visualfield region of the user, the image data piece being one of a pluralityof image data pieces; determining, based on the image data piece,whether an object exists in the blind region; calculating, based on adetermination that an object exists in the blind region, a speed atwhich the object is moving based on the plurality of image data pieces;generating, based on the line-of-sight information read out from amemory and on a relative speed between the moving object and theassistance device, blind region state information including a degree ofurgency indicating a time to an encounter between the object and theuser, and outputting the blind region state information to the projectorof the wearable device, wherein the output blind region stateinformation includes a first mark and a second mark, each of the firstmark and the second mark being projected in a region of the wearabledevice corresponding to a location of the shielding object, wherein thefirst mark indicates less urgency and extends a first distance out fromthe region corresponding to the location of the shielding object, andwherein the second mark indicates more urgency and extends a seconddistance out from the region corresponding to the location of theshielding object, the second distance being less than the firstdistance.
 11. A non-transitory computer-readable recording medium withan executable program stored thereon, the program causing an assistancedevice enabling communication with a wearable device to execute:acquiring line-of-sight information about a line of sight of a userwearing the wearable device that includes a projector configured to emitlight to the user; acquiring an image data piece obtained by capturingan image of a blind region shielded by a shielding object in a visualfield region of the user, the image data piece being one of a pluralityof image data pieces; determining, based on the image data piece,whether an object exists in the blind region; calculating, based on adetermination that an object exists in the blind region, a speed atwhich the object is moving based on the plurality of image data pieces;generating, based on the line-of-sight information and on a relativespeed between the moving object and the assistance device, blind regionstate information including a degree of urgency indicating a time to anencounter between the object and the user; and outputting the blindregion state information to the projector of the wearable device,wherein the output blind region state information includes a first markand a second mark, each of the first mark and the second mark beingprojected in a region of the wearable device corresponding to a locationof the shielding object, wherein the first mark indicates less urgencyand extends a first distance out from the region corresponding to thelocation of the shielding object, and wherein the second mark indicatesmore urgency and extends a second distance out from the regioncorresponding to the location of the shielding object, the seconddistance being less than the first distance.
 12. An assistance deviceenabling communication with a wearable device, the assistance devicecomprising: a memory; and a processor including hardware, the processorbeing configured to: acquire line-of-sight information about a line ofsight of a user wearing the wearable device that includes a projectorconfigured to emit light to the user; acquire an image data pieceobtained by capturing an image of a blind region shielded by a shieldingobject in a visual field region of the user, the image data piece beingone of a plurality of image data pieces; determine, based on the imagedata piece, whether an object exists in the blind region; calculate,based on a determination that an object exists in the blind region, aspeed at which the object is moving based on the plurality of image datapieces; acquire object information from a server; generate, based on theline-of-sight information, the object information, and on a relativespeed between the moving object and the assistance device, blind regionstate information including a degree of urgency indicating a time to anencounter between the object and the user; output the blind region stateinformation to the projector of the wearable device; and transmitcollected data to the server for use by other assistance devices,wherein the output blind region state information includes a first markand a second mark, each of the first mark and the second mark beingprojected in a region of the wearable device corresponding to a locationof the shielding object, wherein the first mark indicates less urgencyand extends a first distance out from the region corresponding to thelocation of the shielding object, and wherein the second mark indicatesmore urgency and extends a second distance out from the regioncorresponding to the location of the shielding object, the seconddistance being less than the first distance.
 13. An assistance deviceconfigured to be worn by a user, comprising: a memory; and a processorincluding hardware, wherein the processor is configured to: acquireline-of-sight information about a line of sight of the user wearing awearable device that includes a projector configured to emit light tothe user; acquire an image data piece obtained by capturing an image ofa blind region shielded by a shielding object in a visual field regionof the user, the image data piece being one of a plurality of image datapieces; determine, based on the image data piece, whether an objectexists in the blind region; calculate, based on a determination that anobject exists in the blind region, a speed at which the object is movingbased on the plurality of image data pieces; acquire object informationfrom a server; generate, based on the line-of-sight information, theobject information, and on a relative speed between the moving objectand the assistance device, blind region state information including adegree of urgency indicating a time to an encounter between the objectand the user; output the blind region state information to the projectorof the wearable device; and transmit collected data to the server foruse by other assistance devices, wherein the output blind region stateinformation includes a first mark and a second mark, each of the firstmark and the second mark being projected in a region of the wearabledevice corresponding to a location of the shielding object, wherein thefirst mark indicates less urgency and extends a first distance out fromthe region corresponding to the location of the shielding object, andwherein the second mark indicates more urgency and extends a seconddistance out from the region corresponding to the location of theshielding object, the second distance being less than the firstdistance.
 14. An assistance method performed by an assistance deviceenabling communication with a wearable device, comprising: acquiringline-of-sight information about a line of sight of a user wearing thewearable device that includes a projector configured to emit light tothe user; acquiring an image data piece obtained by capturing an imageof a blind region shielded by a shielding object in a visual fieldregion of the user, the image data piece being one of a plurality ofimage data pieces; determining, based on the image data piece, whetheran object exists in the blind region; calculating, based on adetermination that an object exists in the blind region, a speed atwhich the object is moving based on the plurality of image data pieces;acquiring object information from a server; generating, based on theline-of-sight information, the object information read out from amemory, and on a relative speed between the moving object and theassistance device, blind region state information including a degree ofurgency indicating a time to an encounter between the object and theuser, outputting the blind region state information to the projector ofthe wearable device and transmitting collected data to the server foruse by other assistance devices, wherein the output blind region stateinformation includes a first mark and a second mark, each of the firstmark and the second mark being projected in a region of the wearabledevice corresponding to a location of the shielding object, wherein thefirst mark indicates less urgency and extends a first distance out fromthe region corresponding to the location of the shielding object, andwherein the second mark indicates more urgency and extends a seconddistance out from the region corresponding to the location of theshielding object, the second distance being less than the firstdistance.
 15. A non-transitory computer-readable recording medium withan executable program stored thereon, the program causing an assistancedevice enabling communication with a wearable device to: acquireline-of-sight information about a line of sight of a user wearing thewearable device that includes a projector configured to emit light tothe user; acquire an image data piece obtained by capturing an image ofa blind region shielded by a shielding object in a visual field regionof the user, the image data piece being one of a plurality of image datapieces; determine, based on the image data piece, whether an objectexists in the blind region; calculate, based on a determination that anobject exists in the blind region, a speed at which the object is movingbased on the plurality of image data pieces; acquire object informationfrom a server; generate, based on the line-of-sight information, theobject information, and on a relative speed between the moving objectand the assistance device, blind region state information including adegree of urgency indicating a time to an encounter between the objectand the user; output the blind region state information to the projectorof the wearable device and transmit collected data to the server for useby other assistance devices, wherein the output blind region stateinformation includes a first mark and a second mark, each of the firstmark and the second mark being projected in a region of the wearabledevice corresponding to a location of the shielding object, wherein thefirst mark indicates less urgency and extends a first distance out fromthe region corresponding to the location of the shielding object, andwherein the second mark indicates more urgency and extends a seconddistance out from the region corresponding to the location of theshielding object, the second distance being less than the firstdistance.